A Home Décor expert in your camera

We present a method to give color advice for Home Décor using images of room finishes, such as paint, flooring, or textiles, taken with uncalibrated cameras. Due to variations in color rendering across devices, an object imaged with different cameras will have different color values. Color information can still be accurately retrieved from uncalibrated images taken under uncontrolled lighting conditions with an unknown device and no access to raw data, when a limited number of known reference colors are available in the scene. We demonstrate that the color of any object can be corrected using a number of calibration targets. A colored object is imaged with an appropriate calibration target in the scene. This target is extracted and its color values are used to compute a color correction transform that is applied to the entire image. Our system finds the closest match of the imaged object to a database of color coordinated paints. We can then supply the users with the appropriate color coordinates. Our results were validated by a Home Décor expert. Introduction Digital cameras have never been so common, compact, and affordable. With their integration into cell phones, most of us walk around with a camera at hand at all times. Images can be useful while shopping and it would be very convenient to simply send an image over the internet or via MMS and get expert advice in return for anything from makeup shade [7] to Home Décor. Despite the ease of taking pictures, objective color assessment remains an issue, especially with the low quality of devices generally integrated into cell phones. The same scene imaged with different devices can result in quite different pixel values due to imperfect illuminant compensation and variable camera characteristics. It is impossible to accurately assign a color from a digital image, unless the camera has been previously calibrated. Color properties of objects are fully characterized by their reflectance spectra, i.e. the percentage of light reflected by the object’s surface at each wavelength and incident angle. However, in many applications it is sufficient to only retrieve tristimulus values, which can be achieved using an RGB camera or scanner [4, 13]. Our method only requires a calibrated target to be present in the scene, which is an inexpensive alternative to the use of calibrated devices. The object of interest is imaged together with a reference target, which allows color correcting images independently of the imaging device and illuminant. The extracted calibration target values allow computing a color correction matrix that is scene and camera dependent. This transform is applied to the entire image. The system relies on the assumption that any camera output color image encoding is sRGB [1]. Figure 1 shows an image of a sample and calibration target, before and after color Figure 1. An object is imaged with a calibration target used for the color correction of the object color. The left and right images are the uncorrected and corrected images, respectively correction. Previous work [8] demonstrated the feasibility of this method for retrieving skin color information from a single digital picture taken with an unknown, casually posed consumer camera and under unknown lighting conditions, using solely a calibrated reference target representative of skin tones present in the scene. In the current paper, we investigate the possibility of adapting the method to wider ranges of hues in order to assess the color of virtually any object. We demonstrate the feasibility for Home Décor applications. The system is fed with an image of an interior color to be coordinated, such as textiles or wall coverings. We do not automatically create color harmonies, which is a difficult subjective task. Rather we use an existing database of color coordinated palettes, designed by an interior designer, in which we find the best color palette complementing the object’s color. The color corrected object pixels are extracted, their values converted to CIECAM02 and compared against a database of paint samples to return the closest match. The system then returns a set of paints that complement the closest match and object colors. A Home Décor expert, a professional interior designer, graded the results as very good. State of the art The irradiance falling on a sensor is proportional to the product of E(x,λ ), the spectral power distribution of the illuminant, with S(x,λ ), the reflectance spectra of the object. The camera response ρi(x) of the ith sensor Ri(λ ) at spatial position x = (x,y) can be modeled as ρi(x) = s(x,λ )T ·diag(e(x,λ )) · ri(λ ), i = 1 : n , (1) where the vectors s(x,λ ), ri(λ ), and e(x,λ ) are, respectively, S(x,λ ), Ri(λ ), and E(x,λ ) represented by 31 samples taken over the visible spectral range [11]. diag(e(x,λ )) is a 31× 31 matrix with the vector entries ei(x,λ ) on its diagonal and n is the number of channels of the imaging device. It is not a trivial task to retrieve reflectance values from cam-